🐛 Source Zendesk Support: refactor TicketComments stream #11237

bazarnov · 2022-03-17T16:17:11Z

What

Resolving: #8519

How

edited stream.py by refactoring the TicketComments stream to fetch the data from incremental/ticket_events endpoint
edited TicketComments stream schema to include missing properties, after refactoring

🚨 User Impact 🚨

No impact expected.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub by running the /publish command described here
After the new connector version is published, connector version bumped in the seed directory as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

codecov · 2022-03-17T16:42:08Z

Codecov Report

❗ No coverage uploaded for pull request base (master@3d9f9ec). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master   #11237   +/-   ##
=========================================
  Coverage          ?   75.22%           
=========================================
  Files             ?        3           
  Lines             ?      331           
  Branches          ?        0           
=========================================
  Hits              ?      249           
  Misses            ?       82           
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3d9f9ec...a2bc55c. Read the comment docs.

bazarnov · 2022-03-17T17:13:59Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1999805755
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/1999805755
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  70     17    76%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              275    106    61%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  876    259    70%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_zendesk_support/__init__.py       2      0   100%
source_zendesk_support/streams.py      290     98    66%
source_zendesk_support/source.py        45     25    44%
--------------------------------------------------------
TOTAL                                  337    123    64%

bazarnov · 2022-03-18T11:56:20Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2003958943
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2003958943
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  70     17    76%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              275    106    61%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  876    259    70%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_zendesk_support/__init__.py       2      0   100%
source_zendesk_support/streams.py      300    101    66%
source_zendesk_support/source.py        45     25    44%
--------------------------------------------------------
TOTAL                                  347    126    64%

bazarnov · 2022-03-18T12:51:30Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2004201618
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2004201618
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  70     17    76%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              275    106    61%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  876    259    70%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_zendesk_support/__init__.py       2      0   100%
source_zendesk_support/streams.py      294     96    67%
source_zendesk_support/source.py        45     25    44%
--------------------------------------------------------
TOTAL                                  341    121    65%

sherifnada

some small change requests

docs/integrations/sources/zendesk-support.md

airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

bazarnov · 2022-03-19T01:25:55Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2007186683
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2007186683
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  70     17    76%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              275    106    61%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  876    259    70%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_zendesk_support/__init__.py       2      0   100%
source_zendesk_support/streams.py      284     81    71%
source_zendesk_support/source.py        45     25    44%
--------------------------------------------------------
TOTAL                                  331    106    68%

sherifnada

Thanks for adding those unit tests!~ Just a couple more requested and we should be good to go

airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

bazarnov · 2022-03-20T13:20:08Z

/test connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2011922667
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2011922667
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        74      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/utils/common.py                  70     17    76%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/tests/test_core.py              275    106    61%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
source_acceptance_test/tests/test_incremental.py        69     38    45%
------------------------------------------------------------------------
TOTAL                                                  876    259    70%
Name                                 Stmts   Miss  Cover
--------------------------------------------------------
source_zendesk_support/__init__.py       2      0   100%
source_zendesk_support/streams.py      284     57    80%
source_zendesk_support/source.py        45     25    44%
--------------------------------------------------------
TOTAL                                  331     82    75%

airbyte-integrations/connectors/source-zendesk-support/unit_tests/test_other.py

bazarnov · 2022-03-21T10:18:49Z

/publish connector=connectors/source-zendesk-support

🕑 connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2015437734
✅ connectors/source-zendesk-support https://github.com/airbytehq/airbyte/actions/runs/2015437734

Fixed the bug when TicketComments stream didn't return all records

a2c4b8c

bazarnov self-assigned this Mar 17, 2022

github-actions bot added the area/connectors Connector related issues label Mar 17, 2022

bazarnov linked an issue Mar 17, 2022 that may be closed by this pull request

🐛 Source Zendesk Support: [ticket comment stream] stream doesn't pull all comments #8519

Closed

added changelog

8072c5e

github-actions bot added the area/documentation Improvements or additions to documentation label Mar 17, 2022

bazarnov requested review from annalvova05 and lazebnyi March 17, 2022 16:18

bazarnov temporarily deployed to more-secrets March 17, 2022 16:20 Inactive

bazarnov temporarily deployed to more-secrets March 17, 2022 16:21 Inactive

formated

0da8663

bazarnov temporarily deployed to more-secrets March 17, 2022 16:27 Inactive

fixed flakeCheck

bc5e314

bazarnov temporarily deployed to more-secrets March 17, 2022 16:32 Inactive

moved the logic to ZendeskSupportTicketEventsStream class

46576cb

bazarnov temporarily deployed to more-secrets March 18, 2022 11:58 Inactive

bazarnov added 2 commits March 18, 2022 14:34

fixed the documentation

70b8c99

fixed the documentation

a1f6a96

bazarnov temporarily deployed to more-secrets March 18, 2022 12:37 Inactive

removed filter_by_state method as not needed

6d3e3e4

bazarnov temporarily deployed to more-secrets March 18, 2022 12:52 Inactive

bazarnov requested a review from sherifnada March 18, 2022 17:38

sherifnada suggested changes Mar 18, 2022

View reviewed changes

updated after review

b3bf785

bazarnov requested a review from sherifnada March 19, 2022 01:26

bazarnov temporarily deployed to more-secrets March 19, 2022 01:28 Inactive

sherifnada reviewed Mar 19, 2022

View reviewed changes

updated unit_tests based on review

94c1310

bazarnov requested a review from sherifnada March 20, 2022 13:20

bazarnov temporarily deployed to more-secrets March 20, 2022 13:22 Inactive

sherifnada approved these changes Mar 21, 2022

View reviewed changes

airbyte-integrations/connectors/source-zendesk-support/unit_tests/test_other.py Outdated Show resolved Hide resolved

bazarnov added 2 commits March 21, 2022 12:17

changed the name of the unit_test file

4ef4aa8

changed the name of the unit_test file

91a3673

bazarnov temporarily deployed to more-secrets March 21, 2022 10:21 Inactive

new spec

a2bc55c

bazarnov temporarily deployed to more-secrets March 21, 2022 11:34 Inactive

bazarnov merged commit 94a862b into master Mar 21, 2022

bazarnov deleted the bazarnov/source-zendesk-support-fix-ticket-comments branch March 21, 2022 11:51

octavia-squidington-iii mentioned this pull request Mar 21, 2022

Bump Airbyte version from 0.35.58-alpha to 0.35.59-alpha #11298

Merged

girarda mentioned this pull request Apr 2, 2022

Source Zendesk Support: Delete unused field and update links to dec #11671

Merged

lazebnyi removed their request for review May 30, 2022 10:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Source Zendesk Support: refactor TicketComments stream #11237

🐛 Source Zendesk Support: refactor TicketComments stream #11237

bazarnov commented Mar 17, 2022 •

edited

Loading

codecov bot commented Mar 17, 2022 •

edited

Loading

bazarnov commented Mar 17, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

sherifnada left a comment

bazarnov commented Mar 19, 2022 •

edited by github-actions bot

Loading

sherifnada left a comment

bazarnov commented Mar 20, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 21, 2022 •

edited by github-actions bot

Loading

🐛 Source Zendesk Support: refactor TicketComments stream #11237

🐛 Source Zendesk Support: refactor TicketComments stream #11237

Conversation

bazarnov commented Mar 17, 2022 • edited Loading

What

How

🚨 User Impact 🚨

Pre-merge Checklist

Community member or Airbyter

Airbyter

codecov bot commented Mar 17, 2022 • edited Loading

Codecov Report

bazarnov commented Mar 17, 2022 • edited by github-actions bot Loading

bazarnov commented Mar 18, 2022 • edited by github-actions bot Loading

bazarnov commented Mar 18, 2022 • edited by github-actions bot Loading

sherifnada left a comment

Choose a reason for hiding this comment

bazarnov commented Mar 19, 2022 • edited by github-actions bot Loading

sherifnada left a comment

Choose a reason for hiding this comment

bazarnov commented Mar 20, 2022 • edited by github-actions bot Loading

bazarnov commented Mar 21, 2022 • edited by github-actions bot Loading

bazarnov commented Mar 17, 2022 •

edited

Loading

codecov bot commented Mar 17, 2022 •

edited

Loading

bazarnov commented Mar 17, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 18, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 19, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 20, 2022 •

edited by github-actions bot

Loading

bazarnov commented Mar 21, 2022 •

edited by github-actions bot

Loading